Papers with few-shot image classification
Improving Few-Shot Image Classification Using Machine- and User-Generated Natural Language Descriptions (2022.findings-naacl)
Copied to clipboard
| Challenge: | Humans can efficiently learn about new concepts from language descriptions, and we propose a new machine learning model, LIDE, which has a text decoder to generate the descriptions and a decoded text encoder to obtain the text representations of machine-generated descriptions. |
| Approach: | They propose a model with a text decoder to generate the descriptions and a corresponding text encoder to obtain the text representations of machine- or user-generated descriptions. |
| Outcome: | The proposed model outperforms baseline models with machine-generated descriptions and with high-quality user-generated models with high quality explanations. |
Data-Efficient Language Shaped Few-shot Image Classification (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing studies have shown that language is helpful guider for image understanding by neural networks. |
| Approach: | They propose a language-shaped learning method that makes the best use of the few-shot images and the language available only in training. |
| Outcome: | The proposed method outperforms state-of-the-art methods on a few-shot dataset with limited training data. |
Semantic Token Reweighting for Interpretable and Controllable Text Embeddings in CLIP (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Despite the varying significance of textual elements within a sentence depending on the context, efforts to account for variation of importance in constructing text embeddings have been lacking. |
| Approach: | They propose a framework for Semantic Token Reweighting to build Interpretable text embeddings which incorporates controllability as well. |
| Outcome: | The proposed framework improves the text encoding process in CLIP by differentially weighting semantic elements based on contextual importance, enabling finer control over emphasis responsive to data-driven insights and user preferences. |